{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quantum Classification With Feature Engineering\n",
    "\n",
    "This notebook offers several examples of feature engineering used with the [QML library](https://docs.microsoft.com/azure/quantum/user-guide/libraries/machine-learning/intro) and a set of exercises on picking the right parameters to distinguish classes of increasingly complex shapes.\n",
    "\n",
    "* This Jupyter notebook can be executed on Azure Quantum.\n",
    "* The first part of this tutorial repeats the steps of the [Exploring Quantum Classification Library tutorial](./ExploringQuantumClassificationLibrary.ipynb) with a slightly different data set for the sake of completeness when running on Azure Quantum.\n",
    "* The companion Q# notebook [Inside Quantum Classifiers](./InsideQuantumClassifiers.ipynb) offers a deep dive in the internals of a simple quantum classifier and several exercises on implementing it from scratch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "To start with, execute this cell using Ctrl+Enter (or ⌘+Enter on a Mac). This is necessary to prepare the environment, import the Q# libraries and operations we'll use later in the tutorial, and configure the plotting routines. If any Python packages are reported as missing, install them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import math\n",
    "import random\n",
    "from typing import List\n",
    "\n",
    "import numpy as np\n",
    "from matplotlib import pyplot\n",
    "pyplot.style.use('ggplot')\n",
    "\n",
    "import warnings\n",
    "warnings.simplefilter('ignore')\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# Plotting configuration\n",
    "cases = [(0, 0), (0, 1), (1, 1), (1, 0)]\n",
    "markers = [\n",
    "    '.' if actual == classified else 'X'\n",
    "    for (actual, classified) in cases\n",
    "]\n",
    "colors = ['blue', 'blue', 'red', 'red']\n",
    "\n",
    "# Q# configuration and necessary imports\n",
    "import qsharp\n",
    "qsharp.packages.add(\"Microsoft.Quantum.MachineLearning\")\n",
    "qsharp.reload()\n",
    "\n",
    "print()\n",
    "print(\"Setup complete!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Classes Separable by Lines That Go Through (0, 0)\n",
    "\n",
    "### Artificial Data Generation\n",
    "\n",
    "The first step of solving a classification problem is preparing the training and validation datasets.\n",
    "\n",
    "> In this tutorial we will use artificially generated data of several different types.\n",
    "A real classification problem will load real data instead, but this choice of artificial data allows to construct simple quantum classifiers by hand, which will be helpful for a deep dive in the classifier structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def generate_angular_data(samples_number : int, min : float, max : float, separation_angles : List[float]):\n",
    "    \"\"\"Generates data with 2 features and 2 classes separable by a pair of lines that go through the origin\"\"\"\n",
    "    features = []\n",
    "    labels = []\n",
    "    for i in range(samples_number):\n",
    "        sample = [random.random() * (max - min) + min, random.random() * (max - min) + min]\n",
    "        angle = math.atan(sample[1] / sample[0])\n",
    "        features.append(sample)\n",
    "        labels.append(0 if angle < separation_angles[0] or angle > separation_angles[1] else 1)\n",
    "    \n",
    "    data = { 'Features' : features, 'Labels' : labels }\n",
    "    return data\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_data(features : list, actual_labels : list, classified_labels : list = None, extra_lines : list = None):\n",
    "    \"\"\"Plots the data, labeling it with actual labels if there are no classification results provided, \n",
    "    and with the classification results (indicating their correctness) if they are provided.\n",
    "    \"\"\"\n",
    "    samples = np.array(features)\n",
    "    pyplot.figure(figsize=(8, 8))\n",
    "    for (idx_case, ((actual, classified), marker, color)) in enumerate(zip(cases, markers, colors)):\n",
    "        mask = np.logical_and(np.equal(actual_labels, actual), \n",
    "                              np.equal(actual if classified_labels == None else classified_labels, classified))\n",
    "        if not np.any(mask): continue\n",
    "        pyplot.scatter(\n",
    "            samples[mask, 0], samples[mask, 1],\n",
    "            label=f\"Class {actual}\" if classified_labels == None else f\"Was {actual}, classified {classified}\",\n",
    "            marker=marker, s=300, c=[color],\n",
    "        )\n",
    "    # Add the lines to show the true classes boundaries, if provided\n",
    "    if extra_lines != None:\n",
    "        for line in extra_lines:\n",
    "            pyplot.plot(line[0], line[1], color='gray')\n",
    "    pyplot.legend()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### Q# Training and Classification Code\n",
    "\n",
    "The Q# code in this cell wraps the calls to the training and classification APIs in the QML library. Later in the tutorial we'll use these wrappers to call the APIs for the specific model structures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "microsoft": {
     "language": "qsharp"
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "%%qsharp\n",
    "open Microsoft.Quantum.Convert;\n",
    "open Microsoft.Quantum.Intrinsic;\n",
    "open Microsoft.Quantum.Canon;\n",
    "open Microsoft.Quantum.Arrays;\n",
    "open Microsoft.Quantum.MachineLearning;\n",
    "open Microsoft.Quantum.Math;\n",
    "\n",
    "// The defaul schedule for drawing batches from a set of samples: just use all samples.\n",
    "function DefaultSchedule(samples : Double[][]) : SamplingSchedule {\n",
    "    return SamplingSchedule([\n",
    "        0..Length(samples) - 1\n",
    "    ]);\n",
    "}\n",
    "\n",
    "\n",
    "// Entry point for training a model; takes the data and the classifier structure as the input and produces model parameters and bias.\n",
    "operation TrainClassifierModel(\n",
    "    trainingVectors : Double[][],\n",
    "    trainingLabels : Int[],\n",
    "    classifierStructure : ControlledRotation[],\n",
    "    initialParameters : Double[][]\n",
    ") : (Double[], Double) {\n",
    "    // convert training data and labels into a single data structure\n",
    "    let samples = Mapped(\n",
    "        LabeledSample,\n",
    "        Zipped(trainingVectors, trainingLabels)\n",
    "    );\n",
    "    let (optimizedModel, nMisses) = TrainSequentialClassifier(\n",
    "        Mapped(\n",
    "            SequentialModel(classifierStructure, _, 0.0),\n",
    "            initialParameters\n",
    "        ),\n",
    "        samples,\n",
    "        DefaultTrainingOptions()\n",
    "            w/ LearningRate <- 2.0\n",
    "            w/ Tolerance <- 0.0005,\n",
    "        DefaultSchedule(trainingVectors),\n",
    "        DefaultSchedule(trainingVectors)\n",
    "    );\n",
    "    Message($\"Training complete, found optimal parameters: {optimizedModel::Parameters}, {optimizedModel::Bias} with {nMisses} misses\");\n",
    "    return (optimizedModel::Parameters, optimizedModel::Bias);\n",
    "}\n",
    "\n",
    "\n",
    "// Entry point for using the model to classify the data; takes validation data, classifier structure, model parameters, and biad as inputs.\n",
    "operation ClassifyData(\n",
    "    samples : Double[][],\n",
    "    classifierStructure : ControlledRotation[],\n",
    "    parameters : Double[],\n",
    "    bias : Double\n",
    ") : Int[] {\n",
    "    let tolerance = 0.0005;\n",
    "    let nMeasurements = 10000;\n",
    "    let model = Default<SequentialModel>()\n",
    "        w/ Structure <- classifierStructure\n",
    "        w/ Parameters <- parameters\n",
    "        w/ Bias <- bias;\n",
    "    let probabilities = EstimateClassificationProbabilities(\n",
    "        tolerance, model,\n",
    "        samples, nMeasurements\n",
    "    );\n",
    "    return InferredLabels(model::Bias, probabilities);\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### Classifier Structure Selection\n",
    "\n",
    "Selecting the right classifier structure requires some insight into the structure of the problem. In this case, we'll encode the data in a single qubit and use a model consisting of a single parameterized rotation gate $R_y(\\theta)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "# Not required, but is helpful to provide type information to the Jupyter Notebooks IDE later in the code.\n",
    "TrainModelAngularData: qsharp.QSharpCallable = None\n",
    "ClassifyAngularData: qsharp.QSharpCallable = None\n",
    "TrainModelVerticalData: qsharp.QSharpCallable = None\n",
    "ClassifyVerticalData: qsharp.QSharpCallable = None\n",
    "TrainModelHorizontalData: qsharp.QSharpCallable = None\n",
    "ClassifyHorizontalData: qsharp.QSharpCallable = None\n",
    "TrainModelCircleData: qsharp.QSharpCallable = None\n",
    "ClassifyCircleData: qsharp.QSharpCallable = None\n",
    "TrainModelHyperbolaData: qsharp.QSharpCallable = None\n",
    "ClassifyHyperbolaData: qsharp.QSharpCallable = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "microsoft": {
     "language": "qsharp"
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "%%qsharp\n",
    "// The definition of classifier structure for the case when the data \n",
    "// is linearly separable by lines going through the origin and fits into 1 qubit.\n",
    "function ClassifierStructureAngularData() : ControlledRotation[] {\n",
    "    return [\n",
    "        ControlledRotation((0, []), PauliY, 0)\n",
    "    ];\n",
    "}\n",
    "\n",
    "// The training code for the case when the data is linearly separable by lines going through the origin.\n",
    "operation TrainModelAngularData(\n",
    "    trainingVectors : Double[][],\n",
    "    trainingLabels : Int[],\n",
    "    initialParameters : Double[][]\n",
    ") : (Double[], Double) {\n",
    "    return TrainClassifierModel(trainingVectors, trainingLabels, ClassifierStructureAngularData(), initialParameters);\n",
    "}\n",
    "\n",
    "// The classification code for the case when the data is linearly separable by lines going through the origin.\n",
    "operation ClassifyAngularData(\n",
    "    samples : Double[][],\n",
    "    parameters : Double[],\n",
    "    bias : Double\n",
    ") : Int[] {\n",
    "    return ClassifyData(samples, ClassifierStructureAngularData(), parameters, bias);\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training and Validation\n",
    "\n",
    "Now that the data generation, training, and classification code is ready, we can get to the interesting part: training the model and validating the results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate training and validation data using the same pair of separation angles\n",
    "separation_angles = [-math.pi / 4, math.pi / 4]\n",
    "training_data_angular = generate_angular_data(150, -1, 1, separation_angles)\n",
    "validation_data_angular = generate_angular_data(50, -1, 1, separation_angles)\n",
    "print(\"Training and validation data generated\")\n",
    "\n",
    "def separation_line(angle : float):\n",
    "    if angle < math.pi / 4:\n",
    "        return [[-1, 1], [-math.tan(angle), math.tan(angle)]]\n",
    "    return [[-1/math.tan(angle), 1/math.tan(angle)], [-1, 1]]\n",
    "\n",
    "# Set up lines that show class separation\n",
    "class_separation_lines_angular = list(map(separation_line, separation_angles))\n",
    "    \n",
    "plot_data(training_data_angular['Features'], training_data_angular['Labels'], extra_lines=class_separation_lines_angular)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train the model\n",
    "(parameters, bias) = TrainModelAngularData.simulate(\n",
    "    trainingVectors=training_data_angular['Features'],\n",
    "    trainingLabels=training_data_angular['Labels'],\n",
    "    initialParameters=[[1.0], [2.0]]                   # use several parameter guesses to start training with\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Classify validation data set using training results\n",
    "classified_labels = ClassifyAngularData.simulate(\n",
    "    samples=validation_data_angular['Features'],\n",
    "    parameters=parameters, bias=bias\n",
    ")\n",
    "\n",
    "# Calculate miss rate\n",
    "mask = np.not_equal(validation_data_angular['Labels'], classified_labels)\n",
    "miss_count = np.array(classified_labels)[np.where(mask)].size\n",
    "miss_rate = miss_count / len(classified_labels)\n",
    "print(f\"Miss rate: {miss_rate:0.2%}\")\n",
    "\n",
    "# Plot validation results\n",
    "plot_data(validation_data_angular['Features'], \n",
    "          validation_data_angular['Labels'], classified_labels, class_separation_lines_angular)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## 2. Classes Separable by Vertical or Horizontal Lines\n",
    "\n",
    "Let's now take a look at using quantum classifiers to distinguish classes of other shapes.\n",
    "\n",
    "### 2.1. Classes Separable by a Vertical Line"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def generate_vertically_separated_data(samples_number : int, min : float, max : float, separation_vertical : float):\n",
    "    \"\"\"Generates data with 2 features and 2 classes separable by a vertical line\"\"\"\n",
    "    features = []\n",
    "    labels = []\n",
    "    for i in range(samples_number):\n",
    "        x = random.random() * (max - min) + min\n",
    "        y = random.random() * (max - min) + min\n",
    "        features.append([x, y])\n",
    "        labels.append(0 if x < separation_vertical else 1)\n",
    "    \n",
    "    data = { 'Features' : features, 'Labels' : labels }\n",
    "    return data\n",
    "\n",
    "# Generate training and validation data using the same separation vertical\n",
    "separation_vertical = 0.5\n",
    "training_data_vertical = generate_vertically_separated_data(150, 0, 1, separation_vertical)\n",
    "validation_data_vertical = generate_vertically_separated_data(50, 0, 1, separation_vertical)\n",
    "print(\"Training and validation data generated\")\n",
    "\n",
    "# Set up lines that show class separation\n",
    "class_separation_lines_vertical = [[[separation_vertical, separation_vertical], [0, 1]]]\n",
    "    \n",
    "plot_data(training_data_vertical['Features'], \n",
    "          training_data_vertical['Labels'], extra_lines=class_separation_lines_vertical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Engineering: Split Fanout\n",
    "\n",
    "We'll use feature engineering - adding new features to the data before encoding it into the circuit for training and classification. \n",
    "\n",
    "For these examples, we'll use _split fanout_ preprocessing, for which the resulting data is a tensor product of (concatenation of the left halves of parameters and original features) and (concatenation of the right halves). \n",
    "That is, 2 original features $[x_0, x_1]$ and 2 parameters $[a_0, a_1]$ produce 4 new features $[a_0a_1, a_0x_1, x_0a_1, x_0x_1]$ that will be encoded in a 2-qubit state $(a_1|0\\rangle + x_1|1\\rangle) \\otimes (a_0|0\\rangle + x_0|1\\rangle)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%qsharp\n",
    "\n",
    "// Split fanout is a tensor product of (concatenation of left halves of parameters and input) and (concatenation of right halves)\n",
    "// [auxil[0] * auxil[1], auxil[0] * input[1], input[0] * auxil[1], input[0] * input[1]]\n",
    "// the length of the result is len(auxil) * len(input)\n",
    "function FeaturesSplitFanout (auxil : Double[], input : Double[]) : Double[] {\n",
    "    let halfLa = Length(auxil) / 2;\n",
    "    let halfLi = Length(input) / 2;\n",
    "    let left = auxil[...(halfLa-1)] + input[...(halfLi-1)];\n",
    "    let right = auxil[halfLa...] + input[halfLi...];\n",
    "\n",
    "    mutable ret = [];\n",
    "    for j in 0 .. Length(left) - 1 {\n",
    "        for k in 0 .. Length(right) - 1 {\n",
    "            set ret += [left[j] * right[k]];\n",
    "        }\n",
    "    }\n",
    "\n",
    "    return ret;\n",
    "}\n",
    "\n",
    "// The definition of classifier structure for the case when the data is vertically separable\n",
    "function ClassifierStructureVerticalData() : ControlledRotation[] {\n",
    "    return [\n",
    "        ControlledRotation((0, []), PauliY, 0)\n",
    "    ];\n",
    "}\n",
    "\n",
    "// The training code for the case when the data is vertically separable\n",
    "operation TrainModelVerticalData(\n",
    "    trainingVectors : Double[][],\n",
    "    trainingLabels : Int[],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    initialParameters : Double[][]\n",
    ") : (Double[], Double) {\n",
    "    let engineeredData = Mapped(FeaturesSplitFanout(featureEngineeringParameters, _), trainingVectors);\n",
    "    return TrainClassifierModel(engineeredData, trainingLabels, ClassifierStructureVerticalData(), initialParameters);\n",
    "}\n",
    "\n",
    "// The classification code for the case when the data is linearly separable by lines going through the origin.\n",
    "operation ClassifyVerticalData(\n",
    "    samples : Double[][],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    parameters : Double[],\n",
    "    bias : Double\n",
    ") : Int[] {\n",
    "    let engineeredData = Mapped(FeaturesSplitFanout(featureEngineeringParameters, _), samples);\n",
    "    return ClassifyData(engineeredData, ClassifierStructureVerticalData(), parameters, bias);\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:blue\">Exercise 1.</span> Figure out feature engineering parameters for classes separable by a vertical line\n",
    "\n",
    "In the code cell below, replace `...` with the parameters $[a_0, a_1]$ used with split fanout preprocessing to produce the data for training and classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# TODO: figure out parameters for split fanout feature engineering\n",
    "feature_engineering_params_vertical = [...]\n",
    "\n",
    "# Train the model\n",
    "(parameters, bias) = TrainModelVerticalData.simulate(\n",
    "    trainingVectors=training_data_vertical['Features'],\n",
    "    trainingLabels=training_data_vertical['Labels'],\n",
    "    featureEngineeringParameters=feature_engineering_params_vertical,\n",
    "    initialParameters=[[0.4], [0.6]]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Classify validation data set using training results\n",
    "classified_labels = ClassifyVerticalData.simulate(\n",
    "    samples=validation_data_vertical['Features'],\n",
    "    featureEngineeringParameters=feature_engineering_params_vertical,\n",
    "    parameters=parameters, bias=bias\n",
    ")\n",
    "\n",
    "# Calculate miss rate\n",
    "mask = np.not_equal(validation_data_vertical['Labels'], classified_labels)\n",
    "miss_count = np.array(classified_labels)[np.where(mask)].size\n",
    "miss_rate = miss_count / len(classified_labels)\n",
    "print(f\"Miss rate: {miss_rate:0.2%}\")\n",
    "\n",
    "# Plot validation results\n",
    "plot_data(validation_data_vertical['Features'], validation_data_vertical['Labels'], classified_labels, class_separation_lines_vertical)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2: Classes Separable by Horizontal Lines\n",
    "\n",
    "Let's repeat the same steps for two classes separable by horizontal lines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def generate_horizontally_separated_data(samples_number : int, min : float, max : float, separation_horizontal : float):\n",
    "    \"\"\"Generates data with 2 features and 2 classes separable by horizontal lines\"\"\"\n",
    "    features = []\n",
    "    labels = []\n",
    "    for i in range(samples_number):\n",
    "        x = random.random() * (max - min) + min\n",
    "        y = random.random() * (max - min) + min\n",
    "        features.append([x, y])\n",
    "        labels.append(0 if abs(y) < separation_horizontal else 1)\n",
    "    \n",
    "    data = { 'Features' : features, 'Labels' : labels }\n",
    "    return data\n",
    "\n",
    "# Generate training and validation data using the same separation horizontals\n",
    "separation_horizontal = 0.75\n",
    "training_data_horizontal = generate_horizontally_separated_data(150, -1.5, 1.5, separation_horizontal)\n",
    "validation_data_horizontal = generate_horizontally_separated_data(50, -1.5, 1.5, separation_horizontal)\n",
    "print(\"Training and validation data generated\")\n",
    "\n",
    "# Set up lines that show class separation\n",
    "class_separation_lines_horizontal = [[[-1.5, 1.5], [separation_horizontal, separation_horizontal]],\n",
    "                                     [[-1.5, 1.5], [-separation_horizontal, -separation_horizontal]]]\n",
    "    \n",
    "plot_data(training_data_horizontal['Features'], training_data_horizontal['Labels'], extra_lines=class_separation_lines_horizontal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%qsharp\n",
    "\n",
    "// The definition of classifier structure for the case when the data is horizontally separable.\n",
    "// Three controlled Ry gates with the same parameter approximate a SWAP gate\n",
    "function ClassifierStructureHorizontalData() : ControlledRotation[] {\n",
    "    return [ControlledRotation((0, [1]), PauliY, 0),\n",
    "            ControlledRotation((1, [0]), PauliY, 0),\n",
    "            ControlledRotation((0, [1]), PauliY, 0)];\n",
    "}\n",
    "\n",
    "// The training code for the case when the data is horizontally separable\n",
    "operation TrainModelHorizontalData(\n",
    "    trainingVectors : Double[][],\n",
    "    trainingLabels : Int[],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    initialParameters : Double[][]\n",
    ") : (Double[], Double) {\n",
    "    let engineeredData = Mapped(FeaturesSplitFanout(featureEngineeringParameters, _), trainingVectors);\n",
    "    return TrainClassifierModel(engineeredData, trainingLabels, ClassifierStructureHorizontalData(), initialParameters);\n",
    "}\n",
    "\n",
    "// The classification code for the case when the data is linearly separable by lines going through the origin.\n",
    "operation ClassifyHorizontalData(\n",
    "    samples : Double[][],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    parameters : Double[],\n",
    "    bias : Double\n",
    ") : Int[] {\n",
    "    let engineeredData = Mapped(FeaturesSplitFanout(featureEngineeringParameters, _), samples);\n",
    "    return ClassifyData(engineeredData, ClassifierStructureHorizontalData(), parameters, bias);\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:blue\">Exercise 2.</span> Figure out feature engineering parameters for classes separable by horizontal lines\n",
    "\n",
    "In the code cell below, replace `...` with the parameters $[a_0, a_1]$ used with split fanout preprocessing to produce the data for training and classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# TODO: figure out parameters for split fanout feature engineering\n",
    "feature_engineering_params_horizontal = [...]\n",
    "\n",
    "# Train the model\n",
    "(parameters, bias) = TrainModelHorizontalData.simulate(\n",
    "    trainingVectors=training_data_horizontal['Features'],\n",
    "    trainingLabels=training_data_horizontal['Labels'],\n",
    "    featureEngineeringParameters=feature_engineering_params_horizontal,\n",
    "    initialParameters=[[1.0], [2.0], [3.0]]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false,
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Classify validation data set using training results\n",
    "classified_labels = ClassifyHorizontalData.simulate(\n",
    "    samples=validation_data_horizontal['Features'],\n",
    "    featureEngineeringParameters=feature_engineering_params_horizontal,\n",
    "    parameters=parameters, bias=bias\n",
    ")\n",
    "\n",
    "# Calculate miss rate\n",
    "mask = np.not_equal(validation_data_horizontal['Labels'], classified_labels)\n",
    "miss_count = np.array(classified_labels)[np.where(mask)].size\n",
    "miss_rate = miss_count / len(classified_labels)\n",
    "print(f\"Miss rate: {miss_rate:0.2%}\")\n",
    "\n",
    "# Plot validation results\n",
    "plot_data(validation_data_horizontal['Features'], validation_data_horizontal['Labels'], classified_labels, class_separation_lines_horizontal)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## 3. Classes Separable by Second Degree Curves\n",
    "\n",
    "### 3.1. Classes Separable by a Circle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def generate_circle_separated_data(samples_number : int, min : float, max : float, separation_r : float):\n",
    "    \"\"\"Generates data with 2 features and 2 classes separable by a circle\"\"\"\n",
    "    features = []\n",
    "    labels = []\n",
    "    for i in range(samples_number):\n",
    "        x = random.random() * (max - min) + min\n",
    "        y = random.random() * (max - min) + min\n",
    "        features.append([x, y])\n",
    "        labels.append(0 if x**2 + y**2 < separation_r**2 else 1)\n",
    "    \n",
    "    data = { 'Features' : features, 'Labels' : labels }\n",
    "    return data\n",
    "\n",
    "# Generate training and validation data using the same separation circle\n",
    "separation_r = 0.8\n",
    "training_data_circle = generate_circle_separated_data(150, -1, 1, separation_r)\n",
    "validation_data_circle = generate_circle_separated_data(50, -1, 1, separation_r)\n",
    "print(\"Training and validation data generated\")\n",
    "\n",
    "plot_data(training_data_circle['Features'], training_data_circle['Labels'])\n",
    "pyplot.gca().add_patch(pyplot.Circle((0, 0), 0.8, color='gray', fill=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Engineering: Left Padding\n",
    "\n",
    "For these examples, we'll use _left padding_ preprocessing, for which the resulting data is a concatenation of the parameters array and the original features array. \n",
    "That is, \n",
    "* 2 original features $[x_0, x_1]$ and 1 parameter $[a_0]$ produce 3 new features $[a_0, x_0, x_1]$ that will be encoded in a 2-qubit state $a_0|00\\rangle + x_0|10\\rangle + x_1|01\\rangle$.\n",
    "* 2 original features $[x_0, x_1]$ and 2 parameters $[a_0, a_1]$ produce 4 new features $[a_0, a_1, x_0, x_1]$ that will be encoded in a 2-qubit state $a_0|00\\rangle + a_1|10\\rangle + x_0|01\\rangle + x_1|11\\rangle$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%qsharp\n",
    "\n",
    "// Left padding preprocessing is a concatenation of the parameters array and the original features array. \n",
    "// The length of the result is len(auxil) + len(input)\n",
    "function LeftPaddedFeatures(auxil : Double[], input : Double[]) : Double[] {\n",
    "    return auxil + input;\n",
    "}\n",
    "\n",
    "// The definition of classifier structure for the case when the data is separable by a circle\n",
    "function ClassifierStructureCircleData() : ControlledRotation[] {\n",
    "    return [\n",
    "        ControlledRotation((0, []), PauliY, 0)\n",
    "    ];\n",
    "}\n",
    "\n",
    "// The training code for the case when the data is separable by a circle\n",
    "operation TrainModelCircleData(\n",
    "    trainingVectors : Double[][],\n",
    "    trainingLabels : Int[],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    initialParameters : Double[][]\n",
    ") : (Double[], Double) {\n",
    "    let engineeredData = Mapped(LeftPaddedFeatures(featureEngineeringParameters, _), trainingVectors);\n",
    "    return TrainClassifierModel(engineeredData, trainingLabels, ClassifierStructureCircleData(), initialParameters);\n",
    "}\n",
    "\n",
    "// The classification code for the case when the data is separable by a circle\n",
    "operation ClassifyCircleData(\n",
    "    samples : Double[][],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    parameters : Double[],\n",
    "    bias : Double\n",
    ") : Int[] {\n",
    "    let engineeredData = Mapped(LeftPaddedFeatures(featureEngineeringParameters, _), samples);\n",
    "    return ClassifyData(engineeredData, ClassifierStructureCircleData(), parameters, bias);\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:blue\">Exercise 3.</span> Figure out feature engineering parameters for classes separable by a circle\n",
    "\n",
    "In the code cell below, replace `...` with the parameters $[a_0]$ or $[a_0, a_1]$ used with left padding preprocessing to produce the data for training and classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# TODO: figure out parameters for left padding feature engineering\n",
    "feature_engineering_params_circle = [...]\n",
    "\n",
    "# Train the model\n",
    "(parameters, bias) = TrainModelCircleData.simulate(\n",
    "    trainingVectors=training_data_circle['Features'],\n",
    "    trainingLabels=training_data_circle['Labels'],\n",
    "    featureEngineeringParameters=feature_engineering_params_circle,\n",
    "    initialParameters=[[1.0], [2.0], [3.0]]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Classify validation data set using training results\n",
    "classified_labels = ClassifyCircleData.simulate(\n",
    "    samples=validation_data_circle['Features'],\n",
    "    featureEngineeringParameters=feature_engineering_params_circle,\n",
    "    parameters=parameters, bias=bias\n",
    ")\n",
    "\n",
    "# Calculate miss rate\n",
    "mask = np.not_equal(validation_data_circle['Labels'], classified_labels)\n",
    "miss_count = np.array(classified_labels)[np.where(mask)].size\n",
    "miss_rate = miss_count / len(classified_labels)\n",
    "print(f\"Miss rate: {miss_rate:0.2%}\")\n",
    "\n",
    "# Plot validation results\n",
    "plot_data(validation_data_circle['Features'], validation_data_circle['Labels'], classified_labels)\n",
    "pyplot.gca().add_patch(pyplot.Circle((0, 0), 0.8, color='gray', fill=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### 3.2. Classes Separable by a Hyperbola"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def generate_hyperbola_separated_data(samples_number : int, min : float, max : float, separation_r : float):\n",
    "    \"\"\"Generates data with 2 features and 2 classes separable by a hyperbola\"\"\"\n",
    "    features = []\n",
    "    labels = []\n",
    "    for i in range(samples_number):\n",
    "        x = random.random() * (max - min) + min\n",
    "        y = random.random() * (max - min) + min\n",
    "        features.append([x, y])\n",
    "        labels.append(0 if separation_r**2 + x**2 > y**2 else 1)\n",
    "    \n",
    "    data = { 'Features' : features, 'Labels' : labels }\n",
    "    return data\n",
    "\n",
    "# Generate training and validation data using the same separation circle\n",
    "separation_r = 0.3\n",
    "training_data_hyperbola = generate_hyperbola_separated_data(150, -1, 1, separation_r)\n",
    "validation_data_hyperbola = generate_hyperbola_separated_data(50, -1, 1, separation_r)\n",
    "print(\"Training and validation data generated\")\n",
    "\n",
    "plot_data(training_data_hyperbola['Features'], training_data_hyperbola['Labels'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%qsharp\n",
    "\n",
    "// The definition of classifier structure for the case when the data is separable by a hyperbola\n",
    "function ClassifierStructureHyperbolaData() : ControlledRotation[] {\n",
    "    return [ControlledRotation((0, []), PauliY, 0)];\n",
    "}\n",
    "\n",
    "// The training code for the case when the data is separable by a hyperbola\n",
    "operation TrainModelHyperbolaData(\n",
    "    trainingVectors : Double[][],\n",
    "    trainingLabels : Int[],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    initialParameters : Double[][]\n",
    ") : (Double[], Double) {\n",
    "    let engineeredData = Mapped(LeftPaddedFeatures(featureEngineeringParameters, _), trainingVectors);\n",
    "    return TrainClassifierModel(engineeredData, trainingLabels, ClassifierStructureHyperbolaData(), initialParameters);\n",
    "}\n",
    "\n",
    "// The classification code for the case when the data is separable by a hyperbola\n",
    "operation ClassifyHyperbolaData(\n",
    "    samples : Double[][],\n",
    "    featureEngineeringParameters : Double[],\n",
    "    parameters : Double[],\n",
    "    bias : Double\n",
    ") : Int[] {\n",
    "    let engineeredData = Mapped(LeftPaddedFeatures(featureEngineeringParameters, _), samples);\n",
    "    return ClassifyData(engineeredData, ClassifierStructureHyperbolaData(), parameters, bias);\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:blue\">Exercise 4.</span> Figure out feature engineering parameters for classes separable by a hyperbola\n",
    "\n",
    "In the code cell below, replace `...` with the parameters $[a_0]$ or $[a_0, a_1]$ used with left padding preprocessing to produce the data for training and classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# TODO: figure out parameters for left padding feature engineering\n",
    "feature_engineering_params_hyperbola = [...]\n",
    "\n",
    "# Train the model\n",
    "(parameters, bias) = TrainModelHyperbolaData.simulate(\n",
    "    trainingVectors=training_data_hyperbola['Features'],\n",
    "    trainingLabels=training_data_hyperbola['Labels'],\n",
    "    featureEngineeringParameters=feature_engineering_params_hyperbola,\n",
    "    initialParameters=[[1.0], [2.0], [3.0]]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": [
     "invalid_code"
    ]
   },
   "outputs": [],
   "source": [
    "# Classify validation data set using training results\n",
    "classified_labels = ClassifyHyperbolaData.simulate(\n",
    "    samples=validation_data_hyperbola['Features'],\n",
    "    featureEngineeringParameters=feature_engineering_params_hyperbola,\n",
    "    parameters=parameters, bias=bias\n",
    ")\n",
    "\n",
    "# Calculate miss rate\n",
    "mask = np.not_equal(validation_data_hyperbola['Labels'], classified_labels)\n",
    "miss_count = np.array(classified_labels)[np.where(mask)].size\n",
    "miss_rate = miss_count / len(classified_labels)\n",
    "print(f\"Miss rate: {miss_rate:0.2%}\")\n",
    "\n",
    "# Plot validation results\n",
    "plot_data(validation_data_hyperbola['Features'], validation_data_hyperbola['Labels'], classified_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What's Next?\n",
    "\n",
    "This tutorial covered classifying artificial data, taking advantage of its simple structure. Classifying real data will require more complex models - same as in traditional machine learning.\n",
    "\n",
    "* Check out [introduction to quantum machine learning](https://docs.microsoft.com/azure/quantum/user-guide/libraries/machine-learning/) at Microsoft Quantum Development Kit documentation, which features a more interesting example - classifying half-moons dataset.\n",
    "* [Quantum machine learning samples](https://github.com/microsoft/Quantum/tree/main/samples/machine-learning) offer examples of classifying several more datasets."
   ]
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
